Learning Classifier Systems using the Cognitive Mechanism of Anticipatory Behavioral Control

A classifier system is a machine learning system that learns a collection of rules, called classifiers. Mostly, classifiers can be regarded as simple stimulus-response rules. A first level of learning called credit assignment level, consists of reinforcement learning on these classifiers. A classifier is reinforced in dependence on the result of an interaction between the CS and its environment. A second level that is independent of the first one consists of rule discovery. For that a CS usually uses genetic algorithms that can only use very indirect information about the interaction between the system and the environment in the form of rule strengths. It is often the problem with CSs that hierarchical chunks of classifiers are destroyed when the rule discovery is applied. Therefore in some applications CSs don’t use the rule discovery level or don’t delete classifiers (eg. Riolo 1991). This paper gives an introduction to a new kind of CSs that learn with anticipatory behavioral control. These classifier systems are called anticipatory classifier systems (ACSs). Anticipatory behavioral control is a development of reinforcement learning on stimulus-response units and enables us to learn an internal model of an external environment. The main difference between ACSs and other CSs is that in an ACS the rule discovery level is integrated into the credit assignment level. The rule discovery algorithm of an ACS uses immediate environmental information, i.e. it’s a kind of intentional rule discovery. This is a particular feature of ACSs. For example, there are no problems with hierarchical chunks of classifiers. After the introduction we prove the performance of ACSs by comparing them with other CSs. A simulation of an experiment about the latent learning of rats is then discussed and it is shown that ACSs solve the locality/globality dilemma for reactive classifier systems.